Skip to content

Line up LS8 GM cyear product with ARD _nbart aliases - #1155

Open
jmettes wants to merge 1 commit into
masterfrom
update-gm-cyear-band-alias
Open

Line up LS8 GM cyear product with ARD _nbart aliases#1155
jmettes wants to merge 1 commit into
masterfrom
update-gm-cyear-band-alias

Conversation

@jmettes

@jmettes jmettes commented Feb 9, 2023

Copy link
Copy Markdown
Contributor

LS8 ARD product has been updated to use _nbart prefixes for its fields, as well as adding the previous field name as an alias. Statistician has been updated to reflect this, and for the summary product output to match this arrangement.

Add aliases to the geomedian cyear product as well.

@jmettes
jmettes requested a review from emmaai February 9, 2023 03:24
@codecov-commenter

codecov-commenter commented Feb 9, 2023 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.29%. Comparing base (db80d7b) to head (c7206cf).
⚠️ Report is 1046 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1155   +/-   ##
=======================================
  Coverage   99.29%   99.29%           
=======================================
  Files         121      121           
  Lines         997      997           
=======================================
  Hits          990      990           
  Misses          7        7
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jmettes

jmettes commented Feb 9, 2023

Copy link
Copy Markdown
Contributor Author

FYI, I think this will cause a ruckus with the existing product definition. This config is kind of assuming a reprocess of the whole archive, which will need to happen for cloud buffering anyway.

Because it's renaming the band names x to nbart_x, datacube will get confused because the existing datasets were indexed under x. I think this is different from if it were indexed under nbart_x, then it would understand that x is an alias for that.

I tried it in dev:

$ datacube product update --allow-unsafe https://raw.githubusercontent.com/GeoscienceAustralia/dea-config/c7206cffb23f530750ed8ecc676140628ce1a595/products/baseline_satellite_data/geomedian-au/ga_ls8c_nbart_gm_cyear_3.odc-product.yaml

However, the band names didn't get aligned properly (for example, nbart_red doesn't have alias, and instead red is separate band)

# datacube product show ga_ls8c_nbart_gm_cyear_3
description: Geoscience Australia Landsat Nadir BRDF Adjusted Reflectance Terrain,
    Landsat 8 Geomedian Calendar Year Collection 3
license: CC-BY-4.0
load:
    align:
        x: 0
        y: 0
    crs: EPSG:3577
    resolution:
        x: 30
        y: -30
measurements:
-   aliases:
    - blue
    dtype: int16
    name: nbart_blue
    nodata: -999
    units: '1'
-   aliases:
    - green
    dtype: int16
    name: nbart_green
    nodata: -999
    units: '1'
-   dtype: int16
    name: nbart_red
    nodata: -999
    units: '1'
-   aliases:
    - nir
    dtype: int16
    name: nbart_nir
    nodata: -999
    units: '1'
-   aliases:
    - swir1
    dtype: int16
    name: nbart_swir1
    nodata: -999
    units: '1'
-   aliases:
    - swir2
    dtype: int16
    name: nbart_swir2
    nodata: -999
    units: '1'
-   dtype: float32
    name: sdev
    nodata: NaN
    units: '1'
-   dtype: float32
    name: edev
    nodata: NaN
    units: '1'
-   dtype: float32
    name: bcdev
    nodata: NaN
    units: '1'
-   dtype: int16
    name: count
    nodata: -999
    units: '1'
metadata:
    product:
        name: ga_ls8c_nbart_gm_cyear_3
    properties:
        odc:file_format: GeoTIFF
        odc:product_family: geomedian
metadata_type: eo3
name: ga_ls8c_nbart_gm_cyear_3

Querying for it didn't work anymore either

image

{'nir': {'path': 'ga_ls8c_nbart_gm_cyear_3_x24y24_2021--P1Y_final_nir.tif'}, 'red': {'path': 'ga_ls8c_nbart_gm_cyear_3_x24y24_2021--P1Y_final_red.tif'}, 'blue': {'path': 'ga_ls8c_nbart_gm_cyear_3_x24y24_2021--P1Y_final_blue.tif'}, 'edev': {'path': 'ga_ls8c_nbart_gm_cyear_3_x24y24_2021--P1Y_final_edev.tif'}, 'sdev': {'path': 'ga_ls8c_nbart_gm_cyear_3_x24y24_2021--P1Y_final_sdev.tif'}, 'bcdev': {'path': 'ga_ls8c_nbart_gm_cyear_3_x24y24_2021--P1Y_final_bcdev.tif'}, 'count': {'path': 'ga_ls8c_nbart_gm_cyear_3_x24y24_2021--P1Y_final_count.tif'}, 'green': {'path': 'ga_ls8c_nbart_gm_cyear_3_x24y24_2021--P1Y_final_green.tif'}, 'swir1': {'path': 'ga_ls8c_nbart_gm_cyear_3_x24y24_2021--P1Y_final_swir1.tif'}, 'swir2': {'path': 'ga_ls8c_nbart_gm_cyear_3_x24y24_2021--P1Y_final_swir2.tif'}}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@emmaai emmaai emmaai approved these changes
Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jmettes @codecov-commenter @emmaai